13. Lesson Summary and Looking Beyond

heading

Machine Learning Methods Recap and Looking Beyond

ND320 C3 L3 10 Summary

Vocabulary Terms & Further Research

Congratulations on completing our lesson on practical machine learning methods for 3D medical imaging analysis! The skills that you have learned through watching videos and doing exercises will help you in the final project where you will build your own AI system using a U-net implementation, and apply it to DICOM datasets.

In this lesson, we have covered the following:

  • A quick refresher on how convolutional neural networks operate and a took a closer look at the different types of convolutions that underlie the operation of these networks.
  • Ways to approach segmentation and classification problems for 3D medical imaging
  • We did an exercise where we trained our own segmentation network on a medical imaging dataset
  • Technical methods for evaluating performance of CNNs for 3D medical image analysis, and talked about the clinical aspect of evaluating performance.

Before we are ready to implement the full-scale AI solution in the final project, there is one final set of concepts that I want you to get familiar with - how to integrate such algorithms into real-world systems, and what these real-world systems look like. This would be the topic of our next lesson.

Further Resources

More problems

As mentioned in my closing remarks, machine learning problems in 3D medical imaging do not boil down to only classification and segmentation. The two problems we’ve looked at here help you understand the principles, but there is so much more you can do. Here are some pointers for some amazing things people do with deep neural networks in 3D medical imaging:

Tools and libraries

We tried to minimize the dependency on external libraries and focus on understanding some key concepts. At the same time, there are many tools that the community has developed, which will help you get moving faster with the tasks typical for medical imaging ML workflows.

A few tools/repos worthy of attention are:

Books

Some resources readily available online for free will help you grasp the basic concepts of computer vision and overall machine learning.

  • https://d2l.ai/ - deep learning with a special section on computer vision by Alexander Smola et al. Alexander has a strong history of publications on machine learning algorithms and statistical analysis and is presently serving as a director for machine learning at Amazon Web Services in Palo Alto, CA
  • http://www.mbmlbook.com/ - a book on general concepts of machine learning by Christopher Bishop et al. Christopher has a distinguished career as a machine learning scientist and presently is in charge of Microsoft Research lab in Cambridge, UK, where I had the honor to work on project InnerEye for several years.

More notable papers

Vocabulary

  • Classification - the problem of determining which one of several classes an image belongs to.
  • Object Detection - the problem of finding a (typically rectangular) region within an image that matches with one of several classes of interest.
  • Segmentation - the problem of identifying which specific pixels within an image belong to a certain object of interest.
  • 2D Convolution - an approach to feature extraction where a convolutional filter is applied to a single 2D image.
  • 2.5D Convolution - an approach to feature extraction where 2D convolutions are applied independently to areas around each voxel (either in neighboring planes or in orthogonal planes) and their results are summed up to form a 2D feature map. Such an approach leverages some 3-dimensional information.
  • 3D Convolution - an approach to feature extraction where the convolutional kernel is 3 dimensional and thus combines information from all 3 dimensions into the feature map.
  • Longitudinal follow up - radiological analysis method that involves monitoring how things change over time. These methods are very valuable in, e.g., oncology for tracking slow-growing tumors.
  • Radiation Therapy - one of the methods of treating cancer where the tumor is exposed to ionizing radiation.